Integration of Thermal and Visible Imagery for Robust Foreground Detection in Tele-immersive Spaces by Miles

The creation of tele-immersive environments involves taking parts of the real world, or scene, and synthesizing them in a virtual world. These parts are called objects of interest, and the collection of these objects together form the foreground of the scene. In particular, tele-immersive systems are designed to facilitate communication and collaboration between people. Therefore, the central objects of interest in a tele-immersive system are these people, the things they jointly manipulate, and the tools they need to perform this manipulation. This thesis develops a multi-modal image fusion framework to detect these objects of interest. The framework consists of blob extraction, depth estimation, and coordinate transformations to integrate visible and thermal IR imaging modalities, and results in multi-modal foreground object detection. Experimental results with a prototype tele-immersive system show that fusion of visible and thermal imagery enables robust foreground detection in unstructured environments. The contribution of this work lies in designing the integration framework, prototyping hardware and software components, and evaluating quantitatively the multi-modal foreground detection for a set of standard scenarios.